Baby Mental Life: Study 2

Preregistered analyses

“Baby Mental Life: Study 2” was conducted on MTurk on 2018-08-04.

Our planned sample was 300 participants, and we anticipated that roughly 80% of recruited participants would pass all of our attention checks, so we initially recruited 378 participants (on the idea that ~80% of 378 ~ 300 participants; note that for administrative purposes we need to recuit participants in batches that were divisible by 9). After filtering out participants who failed at least one of our attention checks, we ended up retaining fewer than 300 participants, so we recruited an additional 16 participants for a total of 394 people recruited. At each stage, we recruited women and men through separate studies, in hopes of acquiring a roughly equal split between genders.

In the end, we ended up with a sample of 304 participants who passed our attention checks, 237 of whom came from unique GPS coordinates.

For this first pass, these data INCLUDE participants where there is another participant with an identical set of GPS coordinates as recorded by Qualtrics. Excluding these participants would exclude 67 participants.

Each participant assessed children’s mental capacities at 13 target ages between the ages of 0 and 5 years. For each target, they rated 20 mental capacities on a scale from 0 (not at all capable) to 100 (completely capable).

For more details about the study, see our preregistration here.

NAs introduced by coercionattributes are not identical across measure variables;
they will be droppedJoining, by = "question_qualtrics"

Study 1 EFA

Joining, by = "capacity"
Joining, by = "factor"
Joining, by = "factor"

EFA

To test H1, we planned to conduct an exploratory factor analysis (EFA) collapsing across all 13 target characters (and treating an individual participant’s responses to each character as if they were independent data points) - see the preregistration for more details.

As with Study 1, we planned to examine three factor retention protocols in order to determine how many factors to retain: Parallel analysis, minimizing BIC, and a set of preset criteria outlined in Weisman et al. (2017). Here we look at each solution in turn.

We predicted that we’d see a similar factor structure to that found in Study 1.

Rotation choices

We planned to examine oblimin-rotated solutions (which allow factors to correlate), but you could examine other rotation options by selecting a different rotation type here.

Parallel analysis

How many factors to retain?

Parallel analysis suggests that the number of factors =  4  and the number of components =  4 
Call: fa.parallel(x = d_all, plot = F)
Parallel analysis suggests that the number of factors =  4  and the number of components =  4 

 Eigen Values of 

What are these factors?

Joining, by = "capacity"
Joining, by = "factor"
Joining, by = "factor"

These factors look extremely similar to what we saw in Study 1 (see above). I (Kara) would say that H1 is strongly supported.

Which capacities are attributed to which targets?

We could look at factor scores using the Study 2 EFA to see which capacities were attributed to which targets. This is not the primary way we planned to investigate this - this was listed as a “follow-up analysis” - but I’m putting it here so that it’s in close proximity to the EFA results for ease of interpretation.

Age as numeric (raw)

Ignoring unknown aesthetics: y

Age as numeric (square-root-transformed)

Ignoring unknown aesthetics: y

Age as ordinal

Ignoring unknown aesthetics: y

And here’s a close look at all of the raw data (color-coded according to the Study 2 EFA results):

Joining, by = "capacity"

|=============================                     | 59% ~1 s remaining     
|==============================                    | 60% ~1 s remaining     
|===============================                   | 62% ~1 s remaining     
|================================                  | 64% ~1 s remaining     
|=================================                 | 66% ~1 s remaining     
|==================================                | 68% ~1 s remaining     
|==================================                | 70% ~1 s remaining     
|===================================               | 72% ~1 s remaining     
|====================================              | 73% ~1 s remaining     
|=====================================             | 75% ~1 s remaining     
|======================================            | 77% ~1 s remaining     
|=======================================           | 79% ~1 s remaining     
|========================================          | 81% ~1 s remaining     
|=========================================         | 83% ~1 s remaining     
|=========================================         | 83% ~1 s remaining     
|==========================================        | 85% ~1 s remaining     
|===========================================       | 87% ~0 s remaining     
|============================================      | 89% ~0 s remaining     
|=============================================     | 91% ~0 s remaining     
|==============================================    | 93% ~0 s remaining     
|===============================================   | 95% ~0 s remaining     
|===============================================   | 96% ~0 s remaining     
|================================================  | 98% ~0 s remaining     
|================================================= |100% ~0 s remaining     

Joining, by = c("capacity", "factor", "order")

Joining, by = "capacity"

Minimizing BIC

How many factors to retain?

Very Simple Structure
Call: vss(x = x, n = n, rotate = rotate, diagonal = diagonal, fm = fm, 
    n.obs = n.obs, plot = plot, title = title, use = use, cor = cor)
VSS complexity 1 achieves a maximimum of 0.85  with  1  factors
VSS complexity 2 achieves a maximimum of 0.96  with  2  factors

The Velicer MAP achieves a minimum of 0.02  with  4  factors 
BIC achieves a minimum of  -153.95  with  8  factors
Sample Size adjusted BIC achieves a minimum of  30.35  with  8  factors

Statistics by number of factors 

What are these factors?

convergence not obtained in GPFoblq. 1000 iterations used.

Joining, by = "capacity"
Joining, by = "factor"
Joining, by = "factor"

A more complex picture, but the first 4 factors look similar to what we get through parallel analysis. (I think something similar happened with Study 1, but we should go back and compare.)

Which capacities are attributed to which targets?

Ignoring unknown aesthetics: y

We’ll skip regression analyses and other plots for now.

Preset retention criteria

How many factors to retain?

[1] "Preset criteria suggest retaining 4 factors"

This gives the same solution as parallel analysis - 4 factors :)

Projecting into Study 1 factor space

I (Kara) made a big mistake in thinking through this: I thought we could project a 20-variable dataset into a 60-variable dataset using the predict.psych() function, but we can’t!

I’ve tried to hack together a way to do this, by replacing all missing values at random (either within the full range of the scale, or around the midpoint, or near 0) - but I haven’t gotten anything to work. As you can see below (replacing missing values at random between 0-5), you see slight increases across all factors, and the most dramatic increase for Factor 4 - but I think this is because that factor is least well-defined in the Study 1 EFA solution? I think we need to focus on analyzing factor scores from our Study 2 EFA. I’m sorry for this mistake!

Regression models in Study 2 factor space

Here’s a multilevel linear regression on these factor scores, with random intercepts and slopes (for target and factor) by participant. Target is coded as numeric, with only the linear contrast.

If we try to run the model above (our planned analysis), we get an error: “Model is nearly unidentifiable: very large eigenvalue.” The error suggests rescaling variables, which solves the problem. Here I’ve re-scaled by divided age in months by 12, to get age in years. Let’s make sure to talk about this.

Linear mixed model fit by REML. t-tests use Satterthwaite's method [
lmerModLmerTest]
Formula: score ~ target_num * factor + (target_num + factor | ResponseId)
   Data: efa_all_par_scores %>% mutate(target_num = target_num/12)

REML criterion at convergence: 26763.9

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-7.6052 -0.4546  0.0520  0.5209  4.4725 

Random effects:
 Groups     Name        Variance Std.Dev. Corr                   
 ResponseId (Intercept) 0.37561  0.6129                          
            target_num  0.01555  0.1247   -0.62                  
            factor1     0.18356  0.4284    0.48 -0.35            
            factor2     0.28893  0.5375   -0.53  0.22 -0.43      
            factor3     0.30770  0.5547    0.21  0.26 -0.11 -0.69
 Residual               0.24497  0.4949                          
Number of obs: 15808, groups:  ResponseId, 304

Fixed effects:
                     Estimate Std. Error         df t value Pr(>|t|)    
(Intercept)        -3.699e-01  3.555e-02  3.030e+02 -10.407  < 2e-16 ***
target_num          2.622e-01  7.575e-03  3.030e+02  34.608  < 2e-16 ***
factor1             8.014e-02  2.622e-02  3.387e+02   3.056  0.00242 ** 
factor2            -2.776e-01  3.216e-02  3.260e+02  -8.633 2.69e-16 ***
factor3             2.634e-01  3.310e-02  3.247e+02   7.957 2.96e-14 ***
target_num:factor1 -5.679e-02  4.324e-03  1.429e+04 -13.133  < 2e-16 ***
target_num:factor2  1.967e-01  4.324e-03  1.429e+04  45.498  < 2e-16 ***
target_num:factor3 -1.867e-01  4.324e-03  1.429e+04 -43.171  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

As we predicted (H2), we see dramatic increases in mental capacity attributions across the age range (main effect of target_num).

And also as we predicted (H1), we see differences across factors in where newborns are perceived to start off: Relative to the grand mean, newborns are perceived to start off with more “negative emotions” (distress, frustration, etc.; main effect of factor1), less/fewer capacities in the domain of “cognition and control” (emotional control, self control, etc.; main effect of factor2), and relatively more “bodily sensations” (pain, fatigue, etc.; main effect of factor3). (We could recode this to look at factor4, or just eyeball it from the plot.) Also as predicted, we see that the perceived changes across age vary dramatically across factors: “negative emotions” are perceived to change relatively less over development, “cognition and control” are perceived to change much more over development, and “bodily sensations” are predicted to chagne relatively less.

This is all very much in line with our preregistered hypotheses :)

Now let’s see what the polynomial effects look like (again, looking at age in years instead of months). As we expected, including all of the polynomial effects as random slopes caused the model not to converge (I think we must be calculating df wrong), so I implemented our preregistered remedy and included only the linear effect as a random slope.

convergence code 1 from bobyqa: bobyqa -- maximum number of function evaluations exceededModel failed to converge with max|grad| = 0.237067 (tol = 0.002, component 1)

Linear mixed model fit by REML. t-tests use Satterthwaite's method [
lmerModLmerTest]
Formula: score ~ poly(target_num, 3) * factor + (poly(target_num, 1) +  
    factor | ResponseId)
   Data: efa_all_par_scores %>% mutate(target_num = target_num/12)

REML criterion at convergence: 23281.6

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-8.4079 -0.4921  0.0054  0.5502  4.9596 

Random effects:
 Groups     Name                Variance Std.Dev. Corr                   
 ResponseId (Intercept)           0.2744  0.5238                         
            poly(target_num, 1) 615.9142 24.8176  -0.39                  
            factor1               0.1866  0.4320   0.44 -0.34            
            factor2               0.2918  0.5402  -0.54  0.22 -0.43      
            factor3               0.3089  0.5558   0.33  0.25 -0.11 -0.69
 Residual                         0.1928  0.4391                         
Number of obs: 15808, groups:  ResponseId, 304

Fixed effects:
                               Estimate Std. Error         df t value
(Intercept)                   5.963e-14  3.024e-02  3.024e+02   0.000
poly(target_num, 3)1          5.197e+01  1.490e+00  3.084e+02  34.891
poly(target_num, 3)2         -2.111e+01  4.391e-01  1.427e+04 -48.078
poly(target_num, 3)3          9.471e+00  4.391e-01  1.427e+04  21.568
factor1                       4.472e-14  2.550e-02  3.028e+02   0.000
factor2                      -3.966e-14  3.157e-02  3.030e+02   0.000
factor3                      -3.801e-14  3.245e-02  3.058e+02   0.000
poly(target_num, 3)1:factor1 -1.126e+01  7.606e-01  1.427e+04 -14.803
poly(target_num, 3)2:factor1  3.958e+00  7.606e-01  1.427e+04   5.204
poly(target_num, 3)3:factor1 -2.147e+00  7.606e-01  1.427e+04  -2.823
poly(target_num, 3)1:factor2  3.900e+01  7.606e-01  1.427e+04  51.281
poly(target_num, 3)2:factor2  2.110e+00  7.606e-01  1.427e+04   2.774
poly(target_num, 3)3:factor2 -6.915e+00  7.606e-01  1.427e+04  -9.092
poly(target_num, 3)1:factor3 -3.701e+01  7.606e-01  1.427e+04 -48.658
poly(target_num, 3)2:factor3  1.306e+01  7.606e-01  1.427e+04  17.167
poly(target_num, 3)3:factor3 -5.433e+00  7.606e-01  1.427e+04  -7.143
                             Pr(>|t|)    
(Intercept)                   1.00000    
poly(target_num, 3)1          < 2e-16 ***
poly(target_num, 3)2          < 2e-16 ***
poly(target_num, 3)3          < 2e-16 ***
factor1                       1.00000    
factor2                       1.00000    
factor3                       1.00000    
poly(target_num, 3)1:factor1  < 2e-16 ***
poly(target_num, 3)2:factor1 1.98e-07 ***
poly(target_num, 3)3:factor1  0.00477 ** 
poly(target_num, 3)1:factor2  < 2e-16 ***
poly(target_num, 3)2:factor2  0.00555 ** 
poly(target_num, 3)3:factor2  < 2e-16 ***
poly(target_num, 3)1:factor3  < 2e-16 ***
poly(target_num, 3)2:factor3  < 2e-16 ***
poly(target_num, 3)3:factor3 9.59e-13 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Correlation matrix not shown by default, as p = 16 > 12.
Use print(x, correlation=TRUE)  or
     vcov(x)     if you need it

convergence code: 1
Model failed to converge with max|grad| = 0.237067 (tol = 0.002, component 1)

Lots to sift through here, but in general we see that the effect of target age on mental capacity attributions definitely has linear, quadratic, and cubic components, all three of which seem to vary substantially across factors. Pretty much all of these differences are “significant” (if you consider |t| > 2 to be “significant”) - for interpretation, I would need to look closer at the plot. Let’s pull it up again here, with blue lines approximating the formula y ~ poly(x, 3):

Ignoring unknown aesthetics: y

We can talk through these interpretations together - but I find the difference between Factor 2 (“cognition & control”) and Factor 4 (“positive emotions”) to be especially interesting!

Demographics

Planning for S3 prereg

Joining, by = "capacity"
Column `capacity` joining character vector and factor, coercing into character vectorJoining, by = "capacity"

Linear mixed model fit by REML. t-tests use Satterthwaite's method [
lmerModLmerTest]
Formula: 
response ~ target_num * domain + (target_num + domain | ResponseId) +  
    (target_num | capacity)
   Data: d_temp_culled %>% mutate(target_num = target_num/12)

REML criterion at convergence: 276364.8

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-5.5999 -0.4567  0.0285  0.5102  5.4324 

Random effects:
 Groups     Name        Variance Std.Dev. Corr                   
 ResponseId (Intercept) 241.394  15.537                          
            target_num   12.017   3.467   -0.58                  
            domain1     115.871  10.764   -0.48  0.49            
            domain2     199.249  14.116   -0.28  0.15 -0.44      
            domain3     202.944  14.246    0.58 -0.32 -0.21 -0.57
 capacity   (Intercept)  98.634   9.931                          
            target_num    4.021   2.005   -0.98                  
 Residual               320.297  17.897                          
Number of obs: 31616, groups:  ResponseId, 304; capacity, 8

Fixed effects:
                   Estimate Std. Error      df t value Pr(>|t|)    
(Intercept)          57.758      3.625   4.530  15.933 3.76e-05 ***
target_num            7.579      0.739   4.647  10.256 0.000229 ***
domain1              33.744      6.117   4.082   5.516 0.004968 ** 
domain2             -45.589      6.140   4.142  -7.425 0.001529 ** 
domain3               7.296      6.141   4.145   1.188 0.298402    
target_num:domain1   -6.308      1.233   3.999  -5.116 0.006907 ** 
target_num:domain2    6.709      1.233   3.999   5.442 0.005541 ** 
target_num:domain3   -1.445      1.233   3.999  -1.172 0.306238    
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Linear mixed model fit by REML. t-tests use Satterthwaite's method [
lmerModLmerTest]
Formula: 
response ~ target_num * domain + (target_num + domain | ResponseId) +  
    (1 | capacity)
   Data: d_temp_culled %>% mutate(target_num = target_num/12)

REML criterion at convergence: 276844.7

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-5.5281 -0.4508  0.0443  0.5054  5.4043 

Random effects:
 Groups     Name        Variance Std.Dev. Corr                   
 ResponseId (Intercept) 241.30   15.534                          
            target_num   12.00    3.464   -0.58                  
            domain1     115.72   10.757   -0.48  0.49            
            domain2     199.10   14.110   -0.28  0.15 -0.44      
            domain3     202.79   14.241    0.58 -0.33 -0.21 -0.57
 capacity   (Intercept)  51.56    7.181                          
 Residual               325.55   18.043                          
Number of obs: 31616, groups:  ResponseId, 304; capacity, 8

Fixed effects:
                     Estimate Std. Error         df t value Pr(>|t|)    
(Intercept)           57.7579     2.6940     5.0546  21.440  3.7e-06 ***
target_num             7.5794     0.2088   303.0006  36.297  < 2e-16 ***
domain1               33.7440     4.4466     4.1692   7.589 0.001368 ** 
domain2              -45.5894     4.4773     4.2856 -10.182 0.000367 ***
domain3                7.2957     4.4786     4.2908   1.629 0.173807    
target_num:domain1    -6.3077     0.1115 30088.9984 -56.590  < 2e-16 ***
target_num:domain2     6.7086     0.1115 30088.9984  60.186  < 2e-16 ***
target_num:domain3    -1.4450     0.1115 30088.9985 -12.963  < 2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Linear mixed model fit by REML. t-tests use Satterthwaite's method [
lmerModLmerTest]
Formula: score ~ target_num * domain + (target_num | ResponseId)
   Data: 
d_temp_culled %>% mutate(target_num = target_num/12) %>% group_by(domain,  
    target_num, ResponseId) %>% summarise(score = mean(response,  
    na.rm = T)) %>% ungroup() %>% distinct()

REML criterion at convergence: 138809.4

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-4.7636 -0.5695  0.0200  0.6602  4.9951 

Random effects:
 Groups     Name        Variance Std.Dev. Corr 
 ResponseId (Intercept) 234.91   15.327        
            target_num   10.57    3.251   -0.58
 Residual               347.42   18.639        
Number of obs: 15808, groups:  ResponseId, 304

Fixed effects:
                     Estimate Std. Error         df  t value Pr(>|t|)    
(Intercept)           57.7579     0.9013   303.0000   64.084   <2e-16 ***
target_num             7.5794     0.2088   302.9998   36.297   <2e-16 ***
domain1               33.7440     0.3446 15194.0000   97.928   <2e-16 ***
domain2              -45.5894     0.3446 15194.0000 -132.304   <2e-16 ***
domain3                7.2957     0.3446 15194.0000   21.173   <2e-16 ***
target_num:domain1    -6.3077     0.1628 15194.0000  -38.735   <2e-16 ***
target_num:domain2     6.7086     0.1628 15194.0000   41.197   <2e-16 ***
target_num:domain3    -1.4450     0.1628 15194.0000   -8.873   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Linear mixed model fit by REML. t-tests use Satterthwaite's method [
lmerModLmerTest]
Formula: response ~ target_num + (1 | ResponseId) + (1 + target_num |  
    capacity)
   Data: 
d_temp_culled %>% mutate(target_num = target_num/12) %>% filter(domain ==  
    "NEG")

REML criterion at convergence: 71637.9

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-4.3024 -0.5181  0.0924  0.6438  4.6977 

Random effects:
 Groups     Name        Variance Std.Dev. Corr 
 ResponseId (Intercept) 588.487  24.259        
 capacity   (Intercept) 102.537  10.126        
            target_num    5.404   2.325   -1.00
 Residual               440.359  20.985        
Number of obs: 7904, groups:  ResponseId, 304; capacity, 2

Fixed effects:
            Estimate Std. Error     df t value Pr(>|t|)  
(Intercept)   65.054      7.301  1.077   8.910   0.0611 .
target_num     6.134      1.651  1.000   3.716   0.1673  
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Linear mixed model fit by REML. t-tests use Satterthwaite's method [
lmerModLmerTest]
Formula: response ~ target_num + (1 | ResponseId) + (1 | capacity)
   Data: 
d_temp_culled %>% mutate(target_num = target_num/12) %>% filter(domain ==  
    "NEG")

REML criterion at convergence: 71757.5

Scaled residuals: 
    Min      1Q  Median      3Q     Max 
-4.1585 -0.5067  0.1344  0.6182  4.5498 

Random effects:
 Groups     Name        Variance Std.Dev.
 ResponseId (Intercept) 588.22   24.253  
 capacity   (Intercept)  46.84    6.844  
 Residual               447.36   21.151  
Number of obs: 7904, groups:  ResponseId, 304; capacity, 2

Fixed effects:
             Estimate Std. Error        df t value Pr(>|t|)    
(Intercept)   65.0535     5.0455    1.1758   12.89   0.0328 *  
target_num     6.1344     0.1509 7597.9999   40.66   <2e-16 ***
---
Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1